Adhered substance detection apparatus

ABSTRACT

An adhered substance detection apparatus of an embodiment includes a calculator and a determiner. The calculator calculates an edge feature for each cell based on edge vectors of pixels within the cell in a captured image, and further calculates a region feature for each unit region based on the calculated edge features of the cells within the unit region. The determiner determines an adherence state of an adhered substance on a lens of a camera based on the region feature. The calculator calculates, as the region feature, number of the cells having an edge strength of zero. When the number of the cells having the zero edge strength is equal to or greater than a predetermined number in a predetermined attention area in the captured image, the determiner determines not to perform a determination for detecting whether the lens of the camera is entirely covered by the adhered substance.

BACKGROUND OF THE INVENTION Field of the Invention

The invention relates to an adhered substance detection apparatus and anadhered substance detection method.

Description of the Background Art

Conventionally, there has been known an adhered substance detectionapparatus that detects an adhered substance adhered to a lens of acamera based on a captured image captured by the camera mounted on avehicle or the like. The adhered substance detection apparatus detectsthe adhered substance, for example, based on a difference betweentime-series captured images.

However, as for a conventional technology, accuracy in detecting anadhered substance further needs to be improved.

SUMMARY OF THE INVENTION

According to one aspect of the invention, an adhered substance detectionapparatus includes a calculator and a determiner. The calculatorcalculates an edge feature for each cell based on edge vectors of pixelswithin the cell. The cell consists of a predetermined number of thepixels in a captured image. The calculator further calculates a regionfeature for each unit region based on the calculated edge features ofthe cells within the unit region. The unit region is a predeterminedregion and consists of a predetermined number of the cells. Thedeterminer determines an adherence state of an adhered substance on alens of a camera based on the region feature. The calculator calculates,as the region feature, number of the cells having an edge strength ofzero. The edge strength is a part of the edge feature. When the numberof the cells having the zero edge strength is equal to or greater than apredetermined number in a predetermined attention area in the capturedimage, the determiner determines not to perform a determination fordetecting whether or not the lens of the camera is in anentirely-covered state in which the lens of the camera is entirelycovered by the adhered substance.

An object of the invention is to provide an adhered substance detectionapparatus and an adhered substance detection method capable of improvingaccuracy in detecting an adhered substance.

These and other objects, features, aspects and advantages of theinvention will become more apparent from the following detaileddescription of the invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an outline of an adhered substance detection methodof the embodiment;

FIG. 1B illustrates an outline of the adhered substance detection methodof the embodiment;

FIG. 1C illustrates an outline of the adhered substance detection methodof the embodiment;

FIG. 2 is a block diagram of the adhered substance detection apparatusof the embodiment;

FIG. 3 illustrates a process of a calculator;

FIG. 4 illustrates the process of the calculator;

FIG. 5 illustrates the process of the calculator;

FIG. 6 illustrates the process of the calculator;

FIG. 7 illustrates a process of a calculator in a modification;

FIG. 8 illustrates the process of the calculator in the modification;

FIG. 9A illustrates a process of a determiner;

FIG. 9B illustrates the process of the determiner;

FIG. 9C illustrates the process of the determiner;

FIG. 9D illustrates the process of the determiner; and

FIG. 10 is a flowchart illustrating a procedure of the process that isperformed by the adhered substance detection apparatus of theembodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, an adhered substance detection apparatus and an adheredsubstance detection method of an embodiment will be described in detailwith reference to the drawings. The invention is not limited to theembodiment described below.

First, with reference to FIGS. 1A to 1C, an outline of the adheredsubstance detection method of the embodiment will be described. FIGS. 1Ato 1C illustrate the outline of the adhered substance detection methodof the embodiment.

As shown in FIG. 1A, for example, a captured image I is captured by avehicle-mounted camera in a state in which snow is on a surface of alens of the vehicle-mounted camera. As an example, below will bedescribed a case in which an adhered substance detection apparatus 1(see FIG. 2) using the adhered substance detection method of theembodiment detects a state in which the surface of the lens of thevehicle-mounted camera is entirely covered by snow (hereinafter referredto as “entirely-covered state”). The adhered substance detectionapparatus 1 detects the entirely-covered state based on a feature(hereinafter referred to also as “edge feature”) relating to a luminancegradient of each pixel of the captured image I.

More specifically, as shown in FIG. 1A, the adhered substance detectionapparatus 1 calculates the edge features of pixels PX within a ROI(Region of Interest) that is a predetermined attention area in thecaptured image I and detects an adherence state of snow based on thecalculated edge features of the pixels PX. The edge feature includes anangle feature and a strength feature. The angle feature is defined as adirection of an edge vector (luminance gradient) (hereinafter referredto also as “edge direction”) of each pixel PX. The strength feature isdefined as a size of the edge vector (hereinafter referred to also as“edge strength”) of each pixel PX.

In order to reduce processing load in image processing, the adheredsubstance detection apparatus 1 uses the edge feature of a cell 100 thatis a group of a predetermined number of the pixels PX. Thus, theprocessing load in the image processing can be reduced. The ROI consistsof one or more unit regions UA each of which is a group of the cells100.

Next, the adhered substance detection apparatus 1 calculates a regionfeature for each of the unit regions UA, based on the edge featuresextracted from the cells 100. In other word, the region feature is astatistical edge feature for each unit region UA. For example, theregion feature includes number of pair regions and a sum of edgestrengths of the pair regions. Here, “pair region” is defined as a pairof the cells 100 that are adjacent to each other and that have the edgedirections opposite to each other. The adhered substance detectionapparatus 1 performs an entirely-covered state determination based onthe region feature.

More specifically, as shown in FIG. 1A, a predetermined number of thecells 100 are arranged in a vertical direction and in a horizontaldirection in each of the unit regions UA in the captured image I. Theadhered substance detection apparatus 1 first calculates the edgefeatures of the cells 100 for each of the unit regions UA (a step S1).The edge feature means the edge direction and the edge strength, asdescribed above.

An edge direction for each of the cells 100 is, as shown in FIG. 1A, arepresentative value for the directions of the edge vectors of thepixels PX within each of the cells 100. The edge direction of each ofthe cells 100 is identified as one of angle groups that have apredetermined angle range. In an example shown in FIG. 1A, the edgedirection for each of the cells 100 is identified as one of up, down,left and right directions each of which has a 90-degree angle range. Anedge strength for each of the cells 100 is a representative value forthe strengths of the edge vectors of the pixels PX within each of thecells 100. A calculation process of the edge feature for each cell 100will be described later in detail with reference to FIGS. 3 and 4.

Next, the adhered substance detection apparatus 1 calculates the regionfeature for each of the unit regions UA based on the edge features ofthe cells 100, calculated in the step S1, within each of the unitregions UA. Here, number of the pair regions described above and a sumof the edge strengths of the pair regions are calculated (a step S2).

Then, the adhered substance detection apparatus 1 performs theentirely-covered state determination based on the calculated number ofthe pair regions and the calculated sum of the edge strengths of thepair regions (a step S3). Here, generally, many pair regions tend to belocated on a boundary (edge) of background of the captured image I orthe pair regions on the edge of the background of the captured image Itend to have a strong feature.

Therefore, when the feature is equal to or smaller than a predeterminedvalue, i.e., when both the number of the pair regions and the sum of theedge strengths of the pair regions are small, as shown in FIG. 1A, theadhered substance detection apparatus 1 determines that the edge of thebackground is unrecognizable (unclear) so that the surface of the lensof the vehicle-mounted camera is entirely covered.

One of similar states to the entirely-covered state is a dark situation,as shown in an upper drawing of FIG. 1B, such as night with no lightnearby so that the background of the captured image I is unrecognizable(unclear) due to darkness (referred to also as “dark case No. 1”), butis not the entirely-covered state. In such a case, the edge in thebackground of the captured image I is unrecognizable. In other words,both number of the pair regions and a sum of the edge strengths of thepair regions are small. Thus, there is a possibility that the adheredsubstance detection apparatus 1 incorrectly determines the case as theentirely-covered state in the foregoing entirely-covered statedetermination.

Here, a lower drawing of FIG. 1B is an image showing the angle groupsvisualized in the captured image I (hereinafter referred to as“visualized angle-group image”). A white region extends in a large areain the visualized angle-group image.

The white region represents one or more cells having no angle (i.e.,zero edge strength). The term “zero edge strength” does not have to meanan absolute zero angle, and may include a state in which an edgestrength is within a predetermined range that is set, for example, indesigning. It is known that the cells having no angle (hereinafterreferred to as “no-angle cell(s)”) generally account for approximatelyless than 10% of an image in a case other than the dark case No. 1.Thus, the adhered substance detection method of this embodiment utilizesthe characteristic of the dark case No. 1 that can be acquired from thevisualized angle-group image, to prevent an erroneous determination indetecting the entirely-covered state.

More specifically, as shown in FIG. 1C, when number of the no-anglecells in the ROI is equal to or greater than a predetermined number, theadhered substance detection method of the embodiment, determines not toperform the entirely-covered state determination because it is difficultto determine the entirely-covered state due to darkness.

Thus, according to the adhered substance detection method of theembodiment, it is possible to incorrectly determine such a state as thedark case No. 1, as the entirely-covered state. In other words, accuracyin detecting an adhered substance can be improved.

In the adhered substance detection method of this embodiment, a processof determining the entirely-covered state is performed for each frame ofthe captured image I. The process derives a determination resultindicative of whether or not the processed frame is determined as theentirely-covered state (e.g. “1” or “−1”). However, as shown in FIG. 1C,when the entirely-covered state determination is not performed, theprocess derives a determination result other than “1” and “−1,” forexample, “0 (zero)” for the frame.

A similar state to the entirely-covered state is not limited to the darkcase No. 1 shown in FIG. 1B. Another case similar state to theentirely-covered state, except the dark case No. 1, will be describedlater with reference to FIGS. 9A to 9D.

The below will be more specifically described a configuration example ofthe adhered substance detection apparatus 1 utilizing the foregoingadhered substance detection method of the embodiment.

FIG. 2 is a block diagram of the adhered substance detection apparatus 1of the embodiment. FIG. 2 illustrates the block diagram only includingconfiguration elements necessary to explain a feature of thisembodiment, and general configuration elements are omitted.

In other words, the configuration elements illustrated in FIG. 2 arefunctional concept, and are not necessarily physically configured asillustrated in FIG. 2. For example, a concrete configuration, such asseparation and integration of functional blocks, is not limited to theconfiguration illustrated in FIG. 2. Some or all functional blocks ofthe adhered substance detection apparatus 1 may be functionally orphysically separated or integrated into an appropriate unit, accordingto a load or a use situation of the functional blocks.

As illustrated in FIG. 2, the adhered substance detection apparatus 1 ofthe embodiment includes a memory 2 and a controller 3. The adheredsubstance detection apparatus 1 is connected to a camera 10 and variousdevices 50.

In FIG. 2, the adhered substance detection apparatus 1 is configuredseparately from the camera 10 and the various devices 50, but is notlimited thereto. The adhered substance detection apparatus 1 may beconfigured as one unit with at least one of the camera 10 and thevarious devices 50.

The camera 10 is, for example, a vehicle-mounted camera including alens, such as a fish-eye lens, and an imaging element, such as a CCD(Charge Coupled Device) and a CMOS (Complementary Metal OxideSemiconductor). The camera 10 is, for example, provided at each positioncapable of capturing front, rear and side images of a vehicle, andoutputs the captured image I to the adhered substance detectionapparatus 1.

The various devices 50 acquire a detection result from the adheredsubstance detection apparatus 1 to perform various control of thevehicle. Some among the various devices 50 are a display that informs auser of an adhered substance on the lens of the camera 10 and instructsthe user to remove the adhered substance from the lens, a removingdevice that removes the adhered substance from the lens by ejecting afluid, air, etc. onto the lens, and a vehicle controller that controlsautonomous driving.

The memory 2 is, for example, a semiconductor memory device, such as aRAM (Random Access Memory) and a flash memory, or a memory device, suchas a hard disk or an optical disk. In an example shown in FIG. 2, thememory 2 stores group information 21 and threshold information 22.

The group information 21 is information relating to the foregoing anglegroups. For example, the group information 21 includes the predeterminedangle range for the angle groups, and the like. The thresholdinformation 22 is information relating to a threshold that is used for adetermination process that is performed by a determiner 33 describedlater. The threshold information 22 includes, for example, thepredetermined number (threshold) for the no-angle cells shown in FIG.1C, and the like.

The controller 3 is a CPU (Central Processing Unit) or an MPU (MicroProcessing Unit), etc. that executes various programs stored in a memoryin the adhered substance detection apparatus 1, using a RAM as a workarea. The controller 3 may be an integrated circuit such as an ASIC(Application Specific Integrated Circuit) or an FPGA (Field ProgrammableGate Array).

The controller 3 includes an acquisition part 31, a calculator 32 andthe determiner 33. The controller 3 executes an information processingfunction and/or produces an effect described below.

The acquisition part 31 acquires the captured image I captured by thecamera 10. The acquisition part 31 performs grayscale processing thatconverts luminance of pixels of the acquired captured image I into graylevels from white to black based on the luminance of the pixels of thecaptured image I, and also performs a smoothing process for the pixelsof the captured image I. Then, the acquisition part 31 outputs thecaptured image I to the calculator 32. An arbitrary smoothing filter,such as an averaging filter and Gaussian filter may be used for thesmoothing process. Further, the grayscale processing and/or thesmoothing process may be omitted.

The calculator 32 calculates the edge feature for each of the cells 100in the captured image I received from the acquisition part 31. Here willbe described, with reference to FIGS. 3 and 4, the calculation processof the edge feature that is performed by the calculator 32.

FIGS. 3 and 4 illustrate the calculation process that is performed bythe calculator 32. As shown in FIG. 3, the calculator 32 first performsan edge detection process for each pixel PX to detect a strength of anedge ex in an X-axis direction (the horizontal direction of the capturedimage I) and a strength of an edge ey in a Y-axis direction (thevertical direction of the captured image I). For the edge detectionprocess, an arbitrary edge detection filter, such as Sobel filter andPrewitt filter, may be used.

Next, the calculator 32 calculates an edge vector V based on thedetected strengths of the edge ex and the edge ey in the X-axisdirection and the Y-axis direction, using a trigonometric function. Thecalculator 32 calculates the edge direction that is an angle θ betweenthe edge vector V and an X axis, and the edge strength that is a lengthL of the edge vector V.

Next, the calculator 32 calculates a representative edge direction valuefor each of the cells 100 based on the calculated edge vectors V of thepixels PX within each cell 100. More specifically, as shown in an upperdrawing of FIG. 4, the calculator 32 categorizes the edge directions ofthe edge vectors V of the pixels PX within each cell 100 into four anglegroups (0) to (3) in up, down, left and right directions. The four anglegroups (0) to (3) are generated by dividing an edge direction range from−180° to 180° into the four up, down, left and right directions to havethe 90-degree angle range each.

More specifically, among the edge directions of the edge vectors V ofthe pixels PX, an edge direction within an angle range from −45° to 45°is categorized as the angle group (0) by the calculator 32; an edgedirection within an angle range from 45° to 135° is categorized as theangle group (1) by the calculator 32; an edge direction within an anglerange from 135° to 180° or −180° to −135° is categorized as the anglegroup (2) by the calculator 32; and an edge direction within an anglerange from −135° to −45° is categorized as the angle group (3) by thecalculator 32.

Then, as shown in a lower drawing of FIG. 4, for each cell 100, thecalculator 32 creates a histogram having bins of the angle groups (0) to(3). Then, when, among frequencies of the bins, a largest frequency isequal to or greater than a predetermined threshold THa, the calculator32 derives the angle group corresponding to the bin having the largestfrequency (in a case of FIG. 4, the angle group (1)), as therepresentative edge direction value for the cell 100.

A frequency of the histogram is calculated by summing the edge strengthsof the pixels PX categorized into a same angle group, among the pixelsPX within the cell 100. As a more specific example, when three pixels PXhaving the edge strengths 10, 20, and 30 respectively, are categorizedin the angle group (0), a bin of the histogram, the frequency of theangle group (0) is 60 that is calculated by adding 10, 20 and 30 of theedge strengths.

Based on the histogram calculated in such a manner, the calculator 32calculates a representative edge strength value for each cell 100. Morespecifically, when a frequency of the bin having the largest frequencyin the histogram is equal to or greater than the predetermined thresholdTHa, the representative edge strength value of the cell 100 is thefrequency of the bin having the largest frequency. In other words, acalculation process of the representative edge strength value performedby the calculator 32 is a calculation process that calculates a featurerelating to the edge strengths corresponding to the representative edgedirection value, within each cell 100.

Meanwhile, when the frequency of the bin having the largest frequency issmaller than the predetermined threshold THa, the calculator 32 regardsthe edge direction for the cell 100 as “invalid,” in other words, “norepresentative edge direction value,” i.e., “no angle” described above.Thus, when variation of the edge directions of the pixels PX is large,calculating an edge direction as a representative value is prevented.

The process in FIGS. 3 and 4 performed by the calculator 32 is only anexample, and if a representative edge direction value can be calculated,any process may be performed. For example, the adhered substancedetection apparatus 1 may calculate an average value of the edgedirections of the pixels PX within each cell 100, and may identify oneof the angle groups (0) to (3) corresponding to the average value as therepresentative edge direction value.

For example, in FIG. 4, the cell 100 is an area of 16 pixels PX that arearranged in a 4×4 matrix. However, number of the pixels PX in one cell100 may be arbitrarily set. Further, number of the pixels PX arranged inthe vertical direction may be different from number of the pixels PXarranged in the horizontal direction, such as a 3×5 matrix.

The calculator 32 in FIG. 2 calculates the region feature for each ofthe unit regions UA based on the calculated edge features of the cells100.

More specifically, the calculator 32 calculates number of pair regions200 and a sum of edge strengths of the pair regions 200 for each of theunit regions UA.

Here will be described, with reference to FIGS. 5 and 6, a calculationprocess, performed by the calculator 32, for the number of the pairregions 200 and the sum of the edge strengths for the pair regions 200.FIGS. 5 and 6 illustrate the calculation process that is performed bythe calculator 32.

FIG. 5 illustrates a case in which two pair regions 200 share no cell100, and FIG. 6 illustrates a case in which two pair regions 200 shareone cell 100.

As shown in FIG. 5, the calculator 32 horizontally and vertically scansa plurality of the cells 100 arranged in the unit region UA in thehorizontal direction and the vertical direction, to detect the pairregion 200. In other words, the calculator 32 extracts, as the pairregion 200, a pair of the cells 100 within the unit region UA that areadjacent to each other and that have the edge directions opposite toeach other.

Then, the calculator 32 calculates the number of the extracted pairregions 200 and the sum of the edge strengths of the extracted pairregions 200. For example, when the extracted two pair regions 200 shareno cell 100, as shown in FIG. 5, the calculator 32 determines that thecalculated number of the extracted pair regions 200 is two andcalculates a sum of the edge strengths of the four cells 100 included inthe two pair regions 200 as the sum of the edge strengths of theextracted pair regions 200.

For example, when the extracted two pair regions 200 share one cell 100,as shown in FIG. 6, the calculator 32 determines that the calculatednumber of the extracted pair regions 200 is two and calculates a sum ofthe edge strengths of the three cells 100 included in the two pairregions 200 as the sum of the edge strengths of the extracted pairregions 200.

The calculator 32 may calculate two or more representative edgedirection values for each of the cells 100 based on a plurality of typesof the angle groups, not only the foregoing angle groups of the “fourup, down, left and right directions” but also, for example, angle groupsof “four oblique directions,” to calculate the region feature. This willbe described with reference to FIGS. 7 and 8. FIGS. 7 and 8 illustrate aprocess performed by the calculator 32 of a modification.

The calculator 32 calculates a first representative edge direction valuebased on the “four up, down, left and right directions” that are firstangle groups. The calculator 32 also calculates a second representativeedge direction value based on the “four oblique directions” that aresecond angle groups, as shown in FIG. 7.

In this case, the calculator 32 categorizes the edge directions of theedge vectors V of the pixels PX within each cell 100 into four anglegroups (4) to (7) that are generated by dividing the angle range from−180° to 180° into the four oblique directions to have a 90-degree anglerange each.

More specifically, among the edge directions of the edge vectors V ofthe pixels PX, an edge direction within an angle range from 0° to 90° iscategorized as the angle group (4) by the calculator 32; an edgedirection within an angle range from 90° to 180° is categorized as theangle group (5) by the calculator 32; an edge direction within an anglerange from −180° to −90° is categorized as the angle group (6) by thecalculator 32; and an edge direction within an angle range from −90° to0° is categorized as the angle group (7) by the calculator 32.

As shown in the lower drawing of FIG. 4, for each cell 100, thecalculator 32 creates a histogram having bins of the angle groups (4) to(7). When, among frequencies of the bins of the generated histogram, alargest frequency is equal to or greater than the predeterminedthreshold THa, the calculator 32 derives the angle group correspondingto the bin having the largest frequency, as the second representativeedge direction value for the cell 100.

Thus, as shown in FIG. 8, the two representative edge direction valuesare calculated for each cell 100. Then, as shown in FIG. 8, thecalculator 32 extracts, as the pair region 200, a pair of the cells 100that are adjacent to each other and one of which has at least one of thefirst and second representative edge direction values opposite to one ofthe first and second representative edge direction values of the other.

In other words, since the calculator 32 calculates the firstrepresentative edge direction value and the second representative edgedirection value for each cell 100, the calculator 32 can extract thepair region 200 that cannot be extracted when the calculator 32 onlycalculates one representative edge direction value for each cell 100.

For example, when the edge directions of the pixels PX are categorizedinto the first angle groups, an edge direction 140° is not opposite toan edge direction −40°. However, when the edge directions of the pixelsPX are categorized into the second angle groups, those two edgedirections are opposite to each other. Therefore, a change of the edgedirections within each cell 100 can be detected more accurately.

With reference back to FIG. 2, the calculator 32 further calculates, asthe region feature, number of the cells 100 having the edge strength, apart of the edge feature, of zero. In other words, the calculator 32calculates number of the no-angle cells described above, an averageluminance value, etc. Then the calculator 32 outputs, to the determiner33, the calculated region feature for each unit region UA.

The determiner 33 performs the entirely-covered state determinationbased on the region feature calculated by the calculator 32, i.e., thenumber of the pair regions 200 and the sum of the edge strengths in thepair regions 200.

However, as shown in FIG. 1C, when the number of the no-angle cells isequal to or greater than the predetermined number in the ROI, i.e., whenthe image satisfies a condition of the dark case No. 1, the determiner33 determines not to perform the entirely-covered state determinationbecause it is difficult to determine the entirely-covered state due todarkness.

When a state is not the dark case No. 1 but is similar to theentirely-covered state, the determiner 33 may perform a process toprevent an incorrect determination. FIGS. 9A to 9D illustrate theprocess performed by the determiner 33. The dark case No. 1 is alreadydescribed above so that another case will be described with reference toFIGS. 9A to 9D.

First, a dark case No. 2 will be described with reference to FIGS. 9A to9B. As shown in an upper drawing of FIG. 9A, the dark case No. 2 is acase of insufficient illumination that causes the background of thecaptured image I to be unclear; however, a dark region in the capturedimage I in the dark case No. 2 is not large as compared to the dark caseNo. 1, because, for example, surroundings of the vehicle are slightlylit by a brake light and the like of the vehicle.

Here, a lower drawing of FIG. 9A illustrates a visualized angle-groupimage of the captured image I in the dark case No. 2. In the visualizedangle-group image, the white (no-angle) cells do not extend in a largearea in the captured image I as compared to the dark case No. 1.However, the no-angle cells are scattered in the dark region of thecaptured image I.

The determiner 33 uses the feature of the dark case No. 2 that isacquired from the visualized angle-group image to prevent an incorrectdetermination in the entirely-covered state determination.

More specifically, as shown in FIG. 9B, when the unit regions UAsatisfying a condition below account for a predetermined percentage orgreater in the ROI, the determiner 33 determines not to perform theentirely-covered state determination because it is difficult todetermine the entirely-covered state due to darkness, like the dark caseNo. 1. The condition is that the unit regions UA i) include apredetermined number or a greater number of the no-angle cells and ii)have an average luminance value lower than a predetermined value. Inother words, when the unit regions UA that i) include the predeterminednumber or a greater number of the no-angle cells and ii) have an averageluminance value lower than the predetermined value, account for thepredetermined percentage or greater in the ROI, the determiner 33determines not to perform the entirely-covered state determination.

Thus, it is possible to prevent from incorrectly determining such astate as the dark case No. 2, as the entirely-covered state. In otherwords, accuracy in detecting an adhered substance can be improved.

Being similar to the dark case No. 1, in the dark case No. 2, thedeterminer 33 does not perform the entirely-covered state so that thedeterminer 33 derives “0” as the determination result.

Next, a dark case No. 3 will be described with reference to FIGS. 9C to9D. As shown in an upper drawing of FIG. 9C, the dark case No. 3 is acase of a blurred edge of the background of the captured image Ibecause, for example, the captured image I was captured by a rear cameraat night, having snow melting agent or the like on the lens that causesthe captured image I to be entirely blurred and a portion of a bumper ofthe vehicle in the captured image I to be whited out by a light of thevehicle.

When the lens is entirely covered by snow, an upper portion of an imagemay be bright, for example, by lights in a tunnel. However, when thelens is entirely covered by snow, an image of which a lower portion isbright is seldom captured because such an image is captured, forexample, when the lens is lit from a road surface.

The determiner 33 uses the feature of the dark case No. 3 to prevent anincorrect determination in the entirely-covered state determinationbased on a difference in average luminance value between an ROI_U and anROI_D that are defined in an upper portion and a lower portion of animage, respectively.

A lower drawing of FIG. 9C is a correlation chart having a Y-axisrepresenting the average luminance value of the upper portion ROI_U ofthe captured image I and an X-axis representing the average luminancevalue of the lower portion ROI_D of the captured image I.

According to the correlation chart, when such a state as the dark caseNo. 3 is incorrectly determined as the entirely-covered state, luminanceof the upper portion and luminance of the lower portion in the image aredisproportionate so that the difference in luminance between the upperportion and the lower portion is concentrated and has a tendency thatthe luminance of the lower portion ROI_D is greater than the luminanceof the upper portion ROI_U by 50 or greater (i.e., the luminance islocated below a dotted line in the chart).

Therefore, as shown in FIG. 9D, when the average luminance value of thelower portion ROI_D is greater than the average luminance value of theupper portion ROI_U by a predetermined value or greater (50 or greaterin the example shown in FIG. 9C), the determiner 33 determines that thelens of the camera 10 is not in the entirely-covered state. In otherwords, the determiner 33 derives a determination result “−1.”

Thus, it is possible to prevent from incorrectly determining such astate as the dark case No. 3, as the entirely-covered state. In otherwords, accuracy in detecting an adhered substance can be improved.

With reference back to FIG. 2, the determiner 33 outputs thedetermination result of the determination to the various devices 50.

Next will be described is a procedure of the process that is performedby the adhered substance detection apparatus 1 of the embodiment, withreference to FIG. 10. FIG. 10 is a flowchart illustrating the procedureof the process that is performed by the adhered substance detectionapparatus 1 of the embodiment. FIG. 10 illustrates the procedure of theprocess that is performed for each frame of the captured image I.

As shown in FIG. 10, first the acquisition part 31 acquires the capturedimage I (a step S101). The acquisition part 31 performs the grayscaleprocessing and the smoothing process for the captured image I.

Next, the calculator 32 calculates the edge feature for each of thecells 100 of the captured image I (a step S102). Further, the calculator32 calculates the region feature for each of the unit regions UA basedon the calculated edge features of the cells (a step S103).

Then, the determiner 33 determines, based on the region featurecalculated by the calculator 32, whether or not the captured image Isatisfies the condition of the dark case No. 1 (a step S104). When thecaptured image I satisfies the condition of the dark case No. 1 (Yes inthe step S104), the determiner 33 derives, for example, “0 (zero)” asthe determination result and moves to a step S109 without performing theentirely-covered state determination.

When the captured image I does not satisfy the condition of the darkcase No. 1 (No in the step S104), the determiner 33 next determineswhether or not the captured image I satisfies the condition of the darkcase No. 2 (a step S105). When the captured image I satisfies thecondition of the dark case No. 2 (Yes in the step S105), the determiner33 derives, for example, “0 (zero)” as the determination result andmoves to the step S109 without performing the entirely-covered statedetermination.

When the captured image I does not satisfy the condition of the darkcase No. 2 (No in the step S105), the determiner 33 determines whetheror not the captured image I satisfies the condition of the dark case No.3 (a step S106). When the captured image I satisfies the condition ofthe dark case No. 3 (Yes in the step S106), the determiner 33 determinesthat the lens of the camera 10 is not in the entirely-covered state (astep S107). Then, the determiner 33 derives, for example, “−1” as thedetermination result.

When the captured image I does not satisfy the condition of the darkcase No. 3 (No in the step S106), the determiner 33 performs theentirely-covered state determination based on the number of the pairregions and the sum of the edge strengths of the pair regions calculatedby the calculator 32 (a step S108).

Then, the determiner 33 outputs the determination result to the variousdevices 50 (the step S109) and ends the process.

As described above, the adhered substance detection apparatus 1 of theembodiment includes the calculator 32 and the determiner 33. Each of thecell 100 consists of the predetermined number of the pixels PX in thecaptured image I. Based on the edge vectors of the predetermined numberof the pixels PX within each of the cells 100, the calculator 32calculates the edge feature for each of the cells 100. Further, each ofthe unit regions UA, a predetermined region, consists of thepredetermined number of the cells 100. Based on the calculated edgefeatures of the cells 100 within each of the unit regions UA, thecalculator 32 calculates the region feature for each of the unit regionsUA. The determiner 33 determines, based on the region feature, a stateof an adhered substance on the lens of the camera 10. The calculator 32calculates, as the region feature, number of the cells 100 having theedge strength, a part of the edge feature, of zero. Further, when thenumber of the cells 100 having the zero edge strength (no angle) isequal to or greater than the predetermined number in the ROI in thecaptured image I, the determiner 33 determines not to perform theentirely-covered state determination for detecting whether or not thelens of the camera 10 is in the entirely-covered state.

Thus, accuracy in detecting an adhered substance can be improved by theadhered substance detection apparatus 1 of the embodiment. Especially,in a case of the foregoing dark case No. 1, an incorrect determinationcan be prevented.

Further, the calculator 32 calculates the average luminance value as theregion feature. When the unit regions UA that i) include thepredetermined number or a greater number of the no-angle cells and ii)have an average luminance value lower than the predetermined value,account for the predetermined percentage or greater in the ROI, thedeterminer 33 determines not to perform the entirely-covered statedetermination.

Thus, accuracy in detecting an adhered substance can be improved by theadhered substance detection apparatus 1 of the embodiment. Especially,an incorrect determination can be prevented in the foregoing dark caseNo. 2.

Further, when the average luminance value of the ROI_D defined in thelower portion of the captured image I is greater than the averageluminance value of the ROI_U defined in the upper portion of thecaptured image I by a predetermined value or greater, the determiner 33determines that the lens of the camera 10 is not in the entirely-coveredstate.

Thus, accuracy in detecting an adhered substance can be improved by theadhered substance detection apparatus 1 of the embodiment. Especially,an incorrect determination can be prevented in the foregoing dark caseNo. 3.

The foregoing embodiment describes an example in which the calculator 32categorizes the edge directions into the four angle groups that isgenerated by dividing the edge direction range from −180° to 180° intothe four directions, to have a 90-degree angle range each. However, theangle range that the angle groups have is not limited to 90°. The edgedirection range from −180° to 180° may be divided, for example, into sixangle groups to have a 60-degree angle range each.

Further, the angle range for the first angle groups may be differentfrom the angle range for the second angle groups. For example, the firstangle groups may have a 90-degree angle range, and the second anglegroups may have a 60-degree angle range. Further, lines dividing thesecond angle groups are displaced by 45° from lines dividing the firstangle groups. However, the displaced angle may be greater or smallerthan 45°.

The captured image I in the foregoing embodiment is, for example, animage captured by a vehicle-mounted camera. However, the captured imageI may be an image captured by, for example, a security camera, a camerainstalled on a street light or the like The captured image I may be anyimage captured by a camera having a lens on which a substance may beadhered.

Further effects and modifications may be derived easily by a personskilled in the art. Thus, broader modes of the present invention may notbe limited to the description and typical embodiment described above.

While the invention has been shown and described in detail, theforegoing description is in all aspects illustrative and notrestrictive. It is therefore understood that numerous othermodifications and variations can be devised without departing from thescope of the invention that is defined by the attached claims andequivalents thereof.

What is claimed is:
 1. An adhered substance detection apparatuscomprising a controller configured to function as: a calculator thatcalculates an edge feature for each cell of a plurality of cells of acaptured image photographed by a camera, each of the cells having aplurality of pixels, the plurality of cells being arranged in aplurality of unit regions, each of the unit regions having a pluralityof the cells, the edge feature of each of the cells being calculatedbased on edge vectors of the pixels within the cell, the calculatorfurther calculating a region feature for each of the unit regions basedon the calculated edge features of the cells within the unit region; anda determiner that determines an adherence state of an adhered substanceon a lens of the camera based on the region features calculated for theunit regions; wherein the calculator calculates, as the region feature,a number of the cells having an edge strength of zero, the edge strengthbeing a part of the edge feature; and when the number of the cellshaving the edge strength of zero is equal to or greater than apredetermined number in a predetermined attention area in the capturedimage, the determiner determines not to perform a determination fordetecting whether or not the lens of the camera is in anentirely-covered state in which the lens of the camera is entirelycovered by the adhered substance.
 2. The adhered substance detectionapparatus according to claim 1, wherein the calculator calculates anaverage luminance value for each of the unit regions, and when the unitregions that i) include at least the predetermined number of the cellshaving the edge strength of zero and ii) have the calculated averageluminance value lower than a predetermined value, account for at least apredetermined percentage of the predetermined attention area, thedeterminer determines not to perform the determination for detectingwhether or not the lens of the camera is in the entirely-covered state.3. The adhered substance detection apparatus according to claim 2,wherein when the average luminance value of a first predeterminedattention area that is defined in a lower portion of the captured imageis greater than the average luminance value of a second predeterminedattention area that is defined in an upper portion of the captured imageby at least a predetermined value, the determiner determines that thelens of the camera is not in the entirely-covered state.
 4. An adheredsubstance detection method, comprising the steps of: (a) calculating, bya controller, an edge feature for each cell of a plurality of cells of acaptured image photographed by a camera, each of the cells having aplurality of pixels, the plurality of cells being arranged in aplurality of unit regions, each of the unit regions having a pluralityof the cells, the edge feature of each of the cells being calculatedbased on edge vectors of the pixels within the cell, and furthercalculating, by the controller, a region feature for each of the unitregions based on the calculated edge features of the cells within theunit region; and (b) determining, by the controller, an adherence stateof an adhered substance on a lens of the camera based on the regionfeatures calculated for the unit regions; wherein the step (a)calculates as the region feature, a number of the cells having an edgestrength of zero, the edge strength being a part of the edge feature;and when the number of the cells having the edge strength of zero isequal to or greater than a predetermined number in a predeterminedattention area in the captured image, the step (b) determines not toperform a determination for detecting whether or not the lens of thecamera is in an entirely-covered state in which the lens of the camerais entirely covered by the adhered substance.